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The Learning Resource Center at Lincoln University, Pennsylvania, provides 
tutoring laboratories that are required for developmental reading, writing, and 
math courses. This article reviews the processes used to plan and determine 
the effectiveness of the tutoring laboratories, including logic models, student 
learning outcomes, and the results of an analysis of the differences between 
pretest and posttest scores using a t test and eta 2 (measure of effect). The 
pretest and posttest scores of students who attended at least six lab sessions 
were examined. The t test results indicated a high level of significance and 
the eta 2 score indicated a moderate to strong effect. Students in the tutoring 
laboratories showed gains in academic skills from the pretest to the posttest, 
supporting the effectiveness of the tutoring laboratories, and the assessment 
process provided accessible information that was used to improve the tutor¬ 
ing laboratory program. 

^l_^Jincoln University, Pennsylvania, 
the first Historically Black College and University (HBCU) in the United 
States, is situated on the East Coast near Philadelphia. The University's 
Learning Resource Center (LRC) provides individual tutoring, student 
workshops, and tutoring laboratories required for success (developmen¬ 
tal) courses in education, English, and mathematics. In 2008, the LRC 
was enhanced through the addition of computers and updated technol¬ 
ogy, and its administrators began to search for an appropriate online 
tutoring program. The goal was to improve the tutoring laboratories, 
foster the engagement of students, and effectively utilize the available 
professional tutors. 
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Online tutoring programs, with available professional tutors to 
guide students, provide an efficient way to reach larger numbers 
of students and still retain individualized instruction and practice. 
An exemplary online tutoring program can also provide reliable service 
to students 24 hours per day, seven days per week, when professional 
or peer tutors are unavailable. Gillen, Lye, and Vaughn (2004) found 
that online tutoring provided an effective use of class time as well as 
opportunities for practice for students. With these considerations in 
mind, the staff of the LRC reviewed online tutoring programs for use 
in the tutoring laboratories, and began the process of implementing 
online tutoring. 

This article examines the incorporation of online tutoring into required 
tutoring laboratories for developmental courses in reading, writing, 
and mathematics, including the evaluation of the effectiveness of the 
laboratories. As part of the self-assessment process, the online tutoring 
programs made available information that was crucial in the evaluation 
of the tutoring laboratories. Students’ gains in academic skills from a 
pretest to a posttest were examined and analyzed, leading to modifica¬ 
tions and improvements in the tutoring laboratory programs. 

Tlitoring Laboratories 

Online tutoring programs proved crucial in the enhancement of the 
tutoring laboratories in the LRC. The year prior to fall 2008 had been 
spent researching online programs in order to make an informed decision 
about which program to implement. At that time, the reading and writing 
tutoring laboratories used paper-and-pencil exercises, and no mathemat¬ 
ics lab existed. The Reading Lab used binders of reading selections with 
multiple-choice questions, and the Writing Lab required students to write 
one paragraph per lab session on prompts devised by the tutors. 

At the time, there were two basic types of online tutoring available: 
online tutoring (which could be either synchronous or asynchronous, 
and involves a live tutor who is matched with a student), and web- 
based tutorials (which consist of content that a student works through 
independently). After reviewing input from the university's academic 
departments and information on a number of online tutoring programs, 
content-type programs were chosen. The nature of the required tutor¬ 
ing laboratories and the need for diagnostic assessments were the main 
factors in this decision. The goal of the required tutoring laboratories 
was to provide review and practice of the content areas covered in the 
students' courses. 

In collaboration with the Education and English Departments, the 
LRC staff chose to implement My Reading Lab (2011a) and My Writing 
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Lab (2011) from Pearson Publishing. The tutors customized the content 
of both programs in the areas requested by the Education and English 
Departments. The Education Department, which provides the devel¬ 
opmental reading courses, requested that the Reading Lab concentrate 
on the topics that are covered in the syllabus and on the department's 
reading comprehension assessment. The English Department requested 
that the Writing Lab focus on grammar while the professors would focus 
on the writing process in the classroom. The Mathematics Department 
recommended the purchase of ALEKS®, an online mathematics program 
published by McGraw-Hill. 

ALEKS *, Assessment and LEarning in Knowledge Spaces, utilizes 
artificial intelligence programming to individualize math problems by 
determining what the student knows and what the student is prepared 
to learn. Software engineers, cognitive scientists, and mathematicians 
assembled by Professor Jean-Claude Falmagne of New York University 
and the University of California (Irvine) developed ALEKS * with the 
support of a National Science Foundation grant. The basis of ALEKS ® 
is Knowledge Space Theory developed through the research of Profes¬ 
sor Falmagne and Professor Jean-Claude Doignon at the University of 
Brussels in the area of mathematical cognitive science (ALEKS®, 2011). 

The online tutoring programs used in the LRC in the reading, writing, 
and math labs provided a computer-scored diagnostic pretest and post¬ 
test, a direct measurement of skills learned, and a solid foundation on 
which to base student learning outcomes and an analysis of effective¬ 
ness. Students entered the online tutoring programs using access codes, 
and professors and tutors received a username and password for free 
access. Each program was tailored to the student's study plan and to his 
or her needs. A management module allowed professors and tutors to 
review students' work. 

Beginning in fall 2008, students accessed online tutoring programs to 
take a diagnostic assessment after which an individualized study pro¬ 
gram was generated for each student. At the end of the semester, each 
student completed a postdiagnostic assessment to determine the amount 
of improvement from pretest to posttest. The results of the diagnostic 
testing were examined, and student learning outcomes were developed. 

Laboratory Tutoring Sessions 

The Reading Lab and Writing Lab weekly tutoring sessions, each last¬ 
ing 50 minutes, began with a 15-minute review mini lesson presented 
by the tutors, after which the students worked on the online program 
in that week's topic area. Each week's designated topic was chosen in 
collaboration with the respective academic departments. The Reading 
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Lab topics included main idea, supporting details, patterns of organiza¬ 
tion, inference, critical thinking, vocabulary, and purpose and tone. The 
sequence of topics for the Writing Lab included thesis statement, point 
of view, revision/concise language, proofreading and parallel structure, 
sentence fragments, run-ons/comma splices, commas, pronouns, verb 
agreement, and parts of speech. The Math Lab mini review topics 
included fractions, operations, decimals, percents, introduction to alge¬ 
bra, real numbers, order of operations, and linear equations. 

The online tutoring programs provided an efficient way to assess the 
effectiveness of the tutoring laboratories. Prior to the online programs, 
students completed paper-and-pencil pretests and posttests that were 
scored by hand, with scoring taking up valuable tutor time that could 
have been best spent with students. Also, the scoring of the pretests 
under the previous system was not completed until near the end of the 
semester, negating any formative assessment value. With the instan¬ 
taneous scoring of the pretests and posttests with the online system, 
students immediately saw their areas of strengths and weaknesses, 
and the tutors adapted the tutoring laboratories to proactively assist the 
students in their areas of need. 

The online tutoring programs also provided individual progress reports 
for each student, as well as class progress reports, that fulfilled grant¬ 
reporting requirements. At the end of each semester information was 
available in the online tutoring programs for a summative evaluation 
and an assessment of effectiveness of the tutoring laboratories. 

Assessment 

The Council for the Advancement of Standards in Higher Education 
(CAS) (2010) provides evaluation guidelines for learning assistance 
programs. The guidelines detail how exemplary learning assistance 
programs systematically and periodically gather data from a variety of 
sources, analyze the data, and use the results to make improvements 
in the program. The CAS self-evaluation process and guidelines made 
available parameters of a high-quality program, in essence a blueprint 
for exemplary services. 

In 2008-2009, the LRC conducted a self-evaluation using the CAS 
guidelines and the resulting recommendations were the basis of the 
present assessment process, including the development and modifica¬ 
tion of student learning outcomes and the use of a SWOT (Strengths, 
Weaknesses, Opportunities, and Threats) analysis and action plan. The 
LRC is currently progressing through a self-assessment in preparation for 
applying for certification of the tutoring laboratories from the National 
Association for Developmental Education. 
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In the LRC, qualitative and quantitative methods that included infor¬ 
mation from all stakeholders were used to evaluate and improve the 
program and to enhance services. The LRC examined quantitative data 
from the online tutoring programs and, additionally, reviewed qualita¬ 
tive information from self-report surveys completed by students and 
professors. After the analysis of the data, changes were implemented to 
the tutoring laboratories: review topics were dropped or added, handouts 
were developed, and additional practice on certain topics was included 
in the laboratories. 

In addition, the establishment of student learning outcomes was 
recommended as a yardstick for measuring the effectiveness of the 
program. The online tutoring programs provided diagnostic pretests and 
posttests on which measureable student learning outcomes were based. 
Each semester, data on student learning outcomes were reviewed and 
the student learning outcomes revised. 

Moreover, accreditation from professional organizations such as the 
Middle States Commission on Higher Education (MSCHE) required 
direct evidence of student learning, including improvement between a 
pretest and a posttest, in order to meet official standards (MSCHE, 2009; 
Suskie, 2009). The online assessments conducted in the LRC fulfilled 
the requirements of the CAS's self-assessment and the standards of the 
MSCHE. 

Student Learning Outcomes 

After completing a self-assessment based on the CAS’s standards and 
guidelines, the LRC staff established student learning outcomes. These 
outcomes were then modified each semester after a review and analysis 
of data from a pretest and posttest, and after an analysis of responses 
from an end-of-semester student survey and a professor survey. 
A graphic organizer, in the form of a table, was created to facilitate the 
development of the student learning outcomes. 

The student learning outcomes were based primarily on an analysis 
of the results of the comparison of the pretest and the posttest that 
shows the increase, or lack thereof, of the skills of the participating 
students. Each semester the student learning outcomes were revised 
after determining the percent improvement from pretest to posttest. 
The lab team then created an action plan consisting of the strategies to 
be used in the following semester to assist students in increasing their 
skills. Table 1 illustrates the fall 2010 student learning outcomes for the 
LRC Writing Lab. 



Table 1 

Writing Lab Student Learning Outcomes 

Student Learning Out- Modified Student 

comes Assessment Method Findings Learning Outcomes Action Plan 

Students will increase Data from the diagnos- For students who attended six or Students will increase Students will improve 
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SWOT Analysis and Action Plan 

In addition to student learning outcomes, the staff conducted a SWOT 
analysis for each program and developed an action plan based on 
published evidence of best practices. This assessment helped the lab 
team develop prescriptions for improving services and increasing the 
effectiveness of those services. For a comprehensive review of the use 
of a SWOT analysis and action plan in a learning assistance center, see 
Fullmer (2009). 

Logic Models 

Logic models—depictions of how a program or organization works— 
facilitate reflection, planning, and communication among an organiza¬ 
tion's employees and stakeholders, leading to a high-quality evaluation 
of the program. The staff of the LRC developed a logic model for the 
LRC using an outcomes approach, developing the short-, medium-, and 
long-term outcomes and impact first, and then proceeded to complete 
the model working backwards. After the outcomes and impact were 
developed, the outputs (activities and participants) were listed, and 
then the inputs (resources) were determined. In addition, each tutor¬ 
ing laboratory team developed a logic model for their lab. Professional 
tutors developed a logic model for their respective labs in order to dis¬ 
cover any gaps in services and to gain a comprehensive understanding 
of the program for planning purposes. This information was used to 
complete a self-evaluation, analyze services, and generate strategies for 
improvements in the effectiveness of services. In this manner, signifi¬ 
cant informed decisions were made concerning goals, objectives, and 
student learning outcomes. 

The creation of logic models for the LRC and the tutoring labs revealed 
gaps in services and programs, and focused attention on the relationship 
between actions and results. With this information, funds were able 
to be efficiently allocated and resources effectively apportioned. The 
LRC professional tutors reported that the realization of the long-term 
outcomes and impact led to increased self-esteem and an awareness of 
the significance of their contribution to the university. In addition, the 
development of the logic models built a shared understanding of the 
LRC and supported cohesiveness of the staff through the engagement 
of all in the design and formation of LRC's logic model. The LRC logic 
model is depicted in Figure 1. 
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Self-Report Surveys 

The professional tutors created end-of-semester surveys for both the 
students and the professors teaching the developmental courses. After 
a review of the surveys completed by the students and the professors, 
the resulting information was used in the assessment of the tutoring 
laboratories. A comprehensive report, completed after each semester, 
provided an opportunity to conduct a self-evaluation and develop ways 
to improve services for students and the Lincoln community. The LRC 
used multiple sources of data and a variety of methods of collection, 
including direct sources of evidence, such as a pretest and posttest, in 
order to develop accurate results. Quantitative and qualitative data were 
collected through the self-report surveys, pretest and posttest scores, and 
course grades. The semester reports can be viewed on the LRC website 
at http://www.lincoln.edu/lrc/index.html. 

Effectiveness of Tutoring 

Students who used the tutoring laboratories and online tutoring pro¬ 
grams were given an opportunity to spend further time on task and 
practice their skills. Brophy (1988) found that additional time on task 
relates to increased learning if the activities are successfully imple¬ 
mented and effectively designed, and if the instruction is proficient. 
Increased practice results in increased knowledge, critical thinking 
skills, and student satisfaction and confidence (Cant & Cooper, 2010). 
Additionally, the tutoring laboratories provided students with a short 
mini lesson to review the material presented in the classroom and to 
provide an opportunity for practice at the end of the mini lesson in a 
brief activity and in the online tutoring program. 

Rlieinheimer, Grace-Odeleye, Francois, and Kusorgbor (2010) found 
that tutoring significantly improved academic performance, including 
increased persistence, increased retention, and increased graduation 
rates. Bloom (1984) found that the most effective method of instruc¬ 
tion was one-on-one (or small groups of two or three) tutoring using 
mastery learning. Bloom's study defined mastery learning as including 
formative testing, feedback, and corrective procedures and found that 
higher order thinking skills, such as analysis, evaluation, and synthesis, 
significantly increased. Additionally, Bloom (1984) found that at the 
beginning of a course, if intense individualized review and relearning 
took place, students' learning increased compared to instruction without 
intensive review and relearning. Using enhanced cues and explanations 
in instruction was also found to increase students' learning. 
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Figure 1 

Learning Resource Center Logic Model Adapted from University of 
Wisconsin-Extension (2003) 
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What we invest: 

University funds 

Title III funds 

Act 101 funds 

Staff: Tutors, Peer Tutors, 
Secretary, Administrators 

Training for staff 

Online programs and 
access codes 

Mini lesson plans and exercises 

Computers 

Rooms and offices 

Collaboration with academic 
departments and administrators 


Outputs 

Activities 

Participation 

What we do: 

Who we reach: 

Tutoring laboratories 

Students 

Online tutoring 

Students' families 

programs 

Faculty 

Review mini lessons 

Administrators 

Individual math 
tutoring 

Alumni 

Individual writing 
tutoring 


Writing portfolio 
assistance 


Revision and proof¬ 
reading assistance 


Workshops 


Summer bridge labs 
and tutoring 



4 


Assumptions: 

Following best practices in our services will help students increase their skills. 
Increased student skills will improve retention and graduation rates. 

Increased graduation rates will help attract students and increase Lincoln's reputation. 
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Short 

Outcomes—Impact 

Medium 

Long 

This semester’s 

Next semester/next year’s 

Results in 4/6 years and 

results: 

results: 

more: 

Students increase their 

Students increase their GPA 

Students successfully 

skills to college level 


graduate Lincoln with skills 


Students persist and return 

for graduate school or the 

Students earn a B or 
above in their course 

the next semester/next year 

fast track in a corporation 


Students become successful 

Lincoln's graduation rate 

Students increase life 

academically, socially, and 

increases 

skills and coping skills 

emotionally at Lincoln 

Lincoln's reputation 

Students adjust to 

Lincoln’s persistence/retention 

increases 

college life 

rate increases 

More and better prepared 
students are attracted to 
Lincoln 

Growth of Lincoln 


External Factors: 

Location 

Economy 

State and Federal funding 

Perception of LRC by Faculty, Administrations, and Students 
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Bonham and Boylan (2011) reviewed and identified initiatives that were 
related to improved success rates in developmental mathematics. They 
found that utilizing all the following strategies were linked to improved 
student success: mastery learning, self-paced learning, mentoring, active 
learning, learning strategies and study skills instruction specifically in 
math, supplemental instruction, contextual learning, problem solving, 
modeling, incorporating coursework with labs and learning assistance 
centers, use of online tutoring programs such as ALEKS®, computerized 
tutorials, individual tutoring in math, small-group instruction, reviews 
for tests, classes held on consecutive days, training and professional 
development for professors and tutors, and assistance with math anxiety 
and attitudes toward math. 

The tutoring laboratories made available to students opportunities to 
review, relearn material, and receive further explanations of concepts. 
This study found that students' academic achievement increased dur¬ 
ing the time period from pretest to posttest. Academic achievement is 
related to the persistence and retention of students in higher education. 
Hodges and White (2001) found that tutoring is a contributing factor to 
the academic success of students. The goal of the LRC is to support the 
persistence and retention of students in their pursuit of a college degree 
by helping students improve their academic skills, which can result in 
higher GPAs as well as increased persistence and retention. 

Online tutorial programs can be effective tools in assisting students 
academically. Online tutoring programs give immediate feedback, 
and immediate feedback has been linked to "a positive effect on both 
metacognitive and cognitive gains [in the use of an intelligent tutoring 
system]” (Saadawi et al., 2010, p. 10). The online tutoring programs also 
generate pretest and posttest assessments and progress reports, keys to 
formative assessment and the summative evaluation of program effec¬ 
tiveness. Computerizing the scoring of the pretest and posttest frees 
up the tutors' time so that more individual interaction with students 
can take place and allows tutors to assist more students during a lab 
session. Such technology engages today’s students, provides hands-on 
activities, and fits in with students' perceptions of up-to-date higher 
education. All of the above can support students in their progress 
toward a college degree. 

The training of tutors has been shown to correlate with an increase 
in students' academic skills. Boylan, Bliss, and Bonham (1997) found 
that tutoring programs in postsecondary education that included the 
training of tutors were significantly related to higher first term GPA, 
higher cumulative GPA, and the retention of students in developmen¬ 
tal courses. In consideration of that evidence, the LRC participates 
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in the International Thtoring Program Certification (ITPC) of the 
College Reading and Learning Association and has earned certification 
at all three levels of the ITPC. The increase in the skills of tutors through 
training has contributed to the high caliber of service to students. 

Purpose of the Study 

This study was part of a self-assessment process of the LRC to ascertain 
the effectiveness of the required tutoring laboratories through determin¬ 
ing whether the students who completed both the pretest and posttest, 
usually the students who spent six or more sessions in the laboratories, 
significantly increased their skills in reading, writing, and mathematics. 
Students who completed both the pretest and posttest usually were the 
students who spent six or more sessions in the lab because each labo¬ 
ratory spends up to five sessions conducting an orientation, a pretest, 
and a posttest. 


Method 

Students who were required to participate in reading, writing, and/ 
or mathematics tutoring laboratories as part of their developmental 
coursework completed a pretest and posttest in an online tutoring pro¬ 
gram: My Reading Lab, My Writing Lab, and/or ALEKS m (mathematics). 
Students were placed into the developmental courses according to their 
math and verbal reasoning SAT scores, as well as a locally administered 
reading comprehension assessment. Some students attended more than 
one lab because they placed into more than one developmental course. 
At this point in time, the respective academic departments are review¬ 
ing the placement process, and the Mathematics Department piloted a 
hand-scored departmental placement test in 2011-2012. 

Participants 

The participants in the study were first-year students who had been 
placed in developmental courses because they did not meet the admis¬ 
sion requirements of the university. Sixty percent of the students were 
female and 40% were male. One hundred percent (408) of the students 
were African American and in their first year of college. One hundred 
and eighty-five students attended the Reading Lab in fall 2010, 313 
students attended the Writing Lab, and 155 attended the Math Lab. 
Students attended the tutoring laboratories in addition to their devel¬ 
opmental courses: Education 097 (Reading and Study Skills), Education 
098 (Critical Reading Skills), English 098 (Basic Writing Skills I), English 
099 (Basic Writing Skills II), MAT 098 (Algebra I), and MAT 099 (Algebra 
and Applications). 
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Students in this study entered the university with differing levels of 
motivation, college preparedness, and perceptions and expectations of 
college life. The professional tutors of the LRC had been trained to use 
strategies to motivate students to devote time and effort to their stud¬ 
ies in order to increase the students' persistence in their postsecondary 
education. Students' motivation may have been a key factor in their 
increase in academic skills, with students of higher motivation attend¬ 
ing more labs and classes and putting more time and effort into their 
coursework. Thus, higher motivated students may have gained greater 
benefits from attending class and the labs, spending more time study¬ 
ing, and successfully completing more assignments. 

To increase student motivation, the tutors greeted the students by 
name, made the students feel comfortable in the lab, and exhibited 
a positive attitude. Thtors were trained to incorporate positive and 
encouraging verbal motivators and practice active listening skills, such 
as questioning, reflection, and summarization. Students' feelings were 
acknowledged and the importance of social relationships, including 
the relationship between the tutor and the student, were recognized. 
Visual and hands-on activities were planned, and colorful pictures of 
lab students and posters, including posters identifying star students, 
graced the walls of the lab. 

The qualifications of the professional tutors included bachelor's 
degrees and master's degrees. All tutors have earned level one certifica¬ 
tion by the College Reading and Learning Association's International 
Tutoring Program Certification Program, and 50% have earned all three 
levels of certification. 


Measurements 

My Writing Lab included a diagnostic pretest and posttest, exercises, 
and an online grade book in which to view students' progress. Students 
attended the Writing Lab for one 50-nrinute session per week. My Writing 
Lab , which was customized in collaboration with the English Depart¬ 
ment, focused on grammar. The mini review lessons presented by the 
professional tutors also focused on grammar at the request of the English 
Department. The professional tutors encouraged and guided students 
through the online tutoring program during the lab session, answering 
questions and providing additional examples. The difference in scores 
between the pretest and the posttest was examined to determine the 
percent of improvement. 

Professional reading tutors conducted a mini review lesson on the topic 
of the week at the beginning of each lab session. Students attended one 
50-nrinute Reading Lab session per week. The topics of the week in the 
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Reading Lab were aligned with the course syllabi. The students then 
read and worked in the online reading tutoring program, My Reading 
Lab , which included reading selections with comprehension questions. 
The professional reading tutors facilitated the students' reading in the 
online tutoring program, providing further explanations of concepts and 
supporting students in their reading. The reading selection component 
provided an assessment of each student's reading level by means of 
Lexiles®, which can be converted into reading grade levels such as the 
Flesch Reading Grade Level (My Reading Lab, 2011b). In this study, the 
Flesch Reading Grade Levels were examined at the beginning of the 
semester and at the end of the semester to determine the improvement 
in reading grade level. 

Professional math tutors presented a mini review lesson, aligned 
with the math course syllabi, at the beginning of each lab. Students 
attended two 50-minute Math Lab sessions per week. After the mini 
review lessons, students worked at their own pace in the online tutoring 
program, ALEKS *. (Due to the nature of the online program, with stu¬ 
dents working at their own pace, students' work in the online program 
may not be congruent with the topic being covered in the classroom.) 
Professional math tutors assisted students in their progress through 
the online program, providing encouragement, detailed examples, and 
additional explanations. A diagnostic pretest, conducted in the first two 
lab sessions, determined the beginning point for each student, and the 
program then presented mathematical problems for each student at the 
level for which he or she was ready to learn. A comprehensive posttest 
assessment, similar to the diagnostic pretest, was conducted during the 
last two sessions of the laboratory. The differences in the scores between 
the pretest and the posttest were examined in this study. 

Procedure 

At the beginning of the semester, students completed an online diag¬ 
nostic pretest in their respective online tutoring programs, and at the 
end of the semester the students completed an online posttest. In the 
Writing Lab and the Math Lab these tests consisted of appropriate ques¬ 
tions based on the material covered. In the Reading Lab, the pretest and 
posttest consisted of a reading selection and comprehension questions. 

Math Lab 

Students in a math-intensive major such as in the sciences, psychol¬ 
ogy, or business were registered in one of two developmental courses 
that required a math tiitoring laboratory: MAT 098 (Algebra I) and MAT 
099 (Algebra and Applications). Students were placed according to their 
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math SAT scores (less than 400 for MAT 098, and 400-450 for MAT 099). 
In the fall of 2010, 38 students in MAT 098 and 65 students in MAT 099 
completed both the pretest and posttest and attended at least six ses¬ 
sions. Math Lab students were required to attend two lab sessions per 
week for a total of 25 sessions. At the discretion of the professor, Math 
Lab attendance counted for about 10% of a student's grade. A Math Lab 
session consisted of a 15- to 20-nrinute mini review lesson on topics in 
the course syllabi and 30 to 35 minutes of practice in the online math 
tutoring program, ALEKSf. Students progressed in the online tutor¬ 
ing program at their own pace. Students completed a comprehensive 
assessment (pretest) in the first two lab sessions and the last two lab 
sessions (posttest). 

Reading Lab 

The Education Department administered a reading comprehension 
assessment, created by a professor, and the students’ scores determined 
their placement in one of two levels of developmental courses, EDU 
097 (Reading and Study Skills) and EDU 098 (Critical Reading Skills). 
Each course level of developmental reading required students to attend 
the Reading Lab. In fall semester, 2010, there were 45 students in 
EDU 097 and 79 students in EDU 098 who completed both the pretest 
and posttest and attended at least six lab sessions. Students in both 
courses were required to attend 11 weekly lab sessions (including an 
orientation session), and attendance counted as 10% of their grade. 
Each laboratory session was 50 minutes long and consisted of approxi¬ 
mately a 15-minute mini review lesson and 35 minutes in the online 
tutoring program, My Reading Lab. The review mini lessons focused 
on topics covered in the course syllabi. At the completion of each 
reading selection and comprehension questions, the online program 
determined each student's Lexile® reading level. The Lexile® reading 
level may be translated into a reading grade level using a conversion 
chart provided by My Reading Lab. 

Writing Lab 

Students in the Writing Lab were registered in one of two developmen¬ 
tal courses, ENG 098 (Basic Writing I) and ENG 099 (Basic Writing II), 
depending on their verbal SAT scores (200-380 for ENG 098, and 390-470 
for ENG 099). In fall semester, 2010, there were 30 ENG 098 students and 
70 ENG 099 students who completed both the pretest and posttest and 
attended at least six sessions. Students in both courses were required 
to attend 12 weekly sessions, including one orientation session, and 
attendance counted as 10% of their course grade. A laboratory session 
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was 50 minutes in length and consisted of an approximately 15-minute 
mini review lesson presented by professional tutors and 35 minutes of 
practice in the online tutoring program, My Writing Lab. The mini review 
lessons and online practice covered various grammar topics based on 
the syllabi of the courses. Students completed the pretest in the first two 
lab sessions and the posttest in the last two lab sessions. 

Data Analysis and Results 

t test of Dependent Samples and Eta 2 

The paired sample t test examines the differences in means between 
groups that are related, and tests the effect of a treatment over time 
(Kiess, 2002, p. 211-215; Ravid, 2000, p. 190). The t test examines the dif¬ 
ference between the pretest and posttest to determine if the difference 
is statistically significant or due to random variation. Although the t test 
determines if the differences between two variables are significant, eta 2 
measures the strength of the relationship between the two variables 
(Ferguson, 2009). The difference between the pretest and the posttest 
maybe statistically significant and not due to random variation (f test), 
but may he of a varying effect size (eta 2 ). The statistical program SSPS 
was used to determine eta 2 , and there has been some discussion that 
SPSS actually determines partial eta 2 rather than eta 2 . Partial eta 2 (etap 2 ) 
may overestimate the effect size (Levine & Hullett, 2002; Olejnik & 
Algina, 2003). 

Ferguson (2009) suggests that a moderate effect size for squared asso¬ 
ciations, such as eta 2 , is .25, and a strong effect size is .64. In this study, 
the effect size ranged from approximately 0.60 to 0.89, indicating that 
about 60% to 89% of the differences between variables could be attrib¬ 
uted to the treatment. Although a result maybe statistically significant, 
it may or may not be practically significant, and practical significance 
must be determined in the context of the study (Ferguson, 2009; Gel- 
man & Stern, 2006). In the context of this study, the resiilts were both 
statistically and practically significant, as well as moderate to strong in 
effect size, because the goal of the treatment was to increase students' 
academic skills in order to positively influence the persistence and 
retention of the students in their pursuit of a college degree. 

The difference between the pretest and posttest scores for students 
who attended six or more laboratory sessions was examined. Data were 
gathered from the entire population of students who attended six or more 
tutoring laboratory sessions, eliminating any sampling error. Scores from 
a pretest and posttest for students in the Writing Lab and Math Lab as 
well as the initial and highest reading grade levels for students in the 
Reading Lab were reviewed. 
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Thble 2 indicates that there was a high level of statistical significance 
for the differences in the pretest and posttest, and a moderate to strong 
effect. For the time interval of the fall 2010 semester, those students who 
attended at least six tutoring lab sessions and took both the pretest and 
posttest improved significantly in skills. Both levels of developmental 
math, Math 098 and Math 099, showed a strong effect size; both levels 
of developmental reading, EDU 097 and EDU 098, showed a moderate 
effect size; and both levels of developmental English, ENG 098 and ENG 
099, showed a strong effect size. 

The t test cannot determine the cause of any significant difference, and 
several factors may have influenced the students' performance between 
the pretest and posttest. The factors that may have influenced students' 
gains in academic achievement include students' work and effort, the 
professors' facilitation of the course, the tutors' facilitation of the lab, and 
time on task in the tutoring laboratories. All of the above factors may 
have contributed to a significant increase in the students' academic skills. 

Limitations 

This study was confined to the fall 2010 semester at a Historically 
Black College or University (HBCU) where the majority of the students 
are African American, most from urban areas on the East Coast and 
nearly all receiving financial aid to attend college. This study needs to 
be repeated over time, in differing semesters, and in other milieus in 
order to assess whether or not the findings of this study can be general¬ 
ized to other situations. 

This study included several treatments over the time interval: students' 
commitment to completing the work and studying, professors' teaching 
and management of the course, and time on task and practice, includ¬ 
ing the tutoring laboratories. It may be that the above factors created a 
synergistic effect that is more powerful than each individual treatment. 
Future studies with an experimental design maybe able to break down 
the treatments into components and determine the contribution of each. 

Future Research 

Many opportunities exist for further research in this area. A study 
with an experimental design may be able to determine causation and 
separate out the contribution of the individual components of the 
treatment factors examined in this study. Studies that can evaluate the 
effectiveness of the separate factors within the treatment time interval 
(student motivation and effort, professors' teaching and management 
of the course, and tutoring laboratories) may be able to determine the 
contribution of each and to discover whether there is a synergistic effect. 



Table 2 

Paired Samples t test and Measure of Effect, Eta 2 
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This study involved a particular student population and context. How¬ 
ever, future studies could involve different student populations, large 
and small colleges and universities, and urban and rural milieus. Time 
of year may be a factor in the motivation of students, specifically if there 
are distracting factors at a particular time of year. Students' motivation 
may be greater in the first semester of a school year versus the second 
semester. The processes examined in this study could be applied to dif¬ 
ferent types of tutoring situations—such as required and not required, 
one-on-one and group, small group and large group, online versus not 
online, etc.—to evaluate the methods used to determine the effective¬ 
ness of these types of tutoring. 

Further research in assessment of tutoring programs is needed to serve 
as a guide for making informed decisions concerning the most effec¬ 
tive programs and strategies, and the allocation of funds and resources. 
Effective programs increase students' skills, which are linked to students' 
increased persistence and retention rates that are correlated to the 
graduation rates of colleges and universities (Rheinheimer et al., 2010). 
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